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Abstract 

Nearly convex sets play important roles in convex analysis, optimization and theory 
of monotone operators. We give a systematic study of nearly convex sets, and con¬ 
struct examples of subdifferentials of lower semicontinuous convex functions whose 
domain or ranges are nonconvex. 
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1 Introduction 


In 1960s Minty and Rockafellar coined nearly convex sets Il22ll25ll . Being a generalization 
of convex sets, the notion of near convexity or almost convexity has been gaining popu- 

* Mathematics, Irving K. Barber School, University of British Columbia Okanagan, Kelowna, British 
Columbia VIV 1V7, Canada. E-mail: sarah.moffat@ubc.ca. 

^Mathematics, Irving K. Barber School, University of British Columbia Okanagan, Kelowna, British 
Columbia VIV 1V7, Canada. E-mail: walaa.moursi@ubc. ca. 

iMathematics, Irving K. Barber School, University of British Columbia Okanagan, Kelowna, British 
Columbia VIV 1V7, Canada. E-mail: shawn.wang@ubc.ca. 


1 



larity in the optimization community, see ||6l[TTl,[T2l[TS[T6l. This can be attributed to the 
applications of generalized convexity in economics problems, see for example, IIT9ll2T]| . 
One reason to study nearly convex sets is that for a proper lower semicontinuous convex 
function its subdifferential domain is always nearly convex Il24l Theorem 23.4, Theorem 
6.1], and the same is true for the domain of each maximally monotone operator Il2^ Theo¬ 
rem 12.41]. Maximally monotone operators are extensively studies recently Iini^l3ll4ll27l. 
Another reason is that to study possibly nonconvex functions, a first endeavor perhaps 
should be to study functions whose epigraphs are nearly convex, see, e.g., IIT^ . 

All these motivate our systematic study of nearly convex sets. Some properties of 
nearly convex sets have been partially studied in ||^ |TTi [T2[ |T3] from different perspec¬ 
tives. The purpose of this paper is to give new proofs to some known results, provide 
further characterizations, and extend known results on calculus, relative interiors, reces¬ 
sion cones, and applications. Although nearly convex sets need not be convex, many 
results on convex sets do extend. We also construct proper lower semicontinuous convex 
functions whose sub differential mappings have domains being neither closed nor open; 
or highly nonconvex. 

We remark that nearly convex was called almost convex in lUTlIT^ITSl . Here, we adopt 
the term nearly convex rather than almost convex because of the relationship with nearly 
equal sets which was noted in ||6|]. Note that this definition of nearly convex does not 
coincide with the one provided in IfTTl Definition 2] and 11121 . where nearly convex is a 
generalization of midpoint convexity. 

The remainder of the paper is organized as follows. Some basic notations and facts 
about convex sets and nearly convex sets are given in Section Section gives new 
characterizations of nearly convex sets. In Section]^ we give calculus of nearly convex 
sets and relative interiors. In Section we study recessions of nearly convex sets. Sec¬ 
tion 1^ is devoted to apply results in Section and Section to study maximality of sum 
of several maximally monotone operators and closedness of nearly convex sets under a 
linear mapping. In Section we construct examples of proper lower semicontinuous 
convex functions with prescribed nearly convex sets being their sub differential domain. 
As early as 1970s, Rockafellar provided a convex function whose subdifferential domain 
is not convex [|24|. We give a detailed analysis of his classical example and use it to gener¬ 
ate new examples with pathological subdifferential domains. Open problems appear in 
Section]^ Appendix A contains some proofs of Section]^ 
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2 Preliminaries 


2.1 Notation and terminology 


Throughout this paper, we work in the Euclidean space IR” with norm || • || and inner 
product (•,•). For a set C C R” let clC denote the closure of C, and aff C the ajfine hull of 
C; that is, the smallest affine set containing C. The key object we shall study is: 


Definition 2.1 (near convexity) A set E C R” is nearly convex if there exists a convex set 
C C R” such that 


CCEC clC. 


Obviously, every convex set is nearly convex, but they are many nearly convex sets which 
are not convex. See Figure]^ for two nearly convex sets. 



Figure 1: A GeoGebra IITtI snapshot. Feft: Neither open nor closed convex set. Right: 
Nearly convex but not convex set 


Note that nearly convex sets do not have nice algebra as convex sets do Il24l Section 3], 
as the following two simple examples illustrate. 

Example 2.2 The nearly convex set C C R^ given by 

C := {(xi,X2) I —1 < xi < 1,X2 > 0} U {(—1,0), (1,0)}. 

has 2C 7^ C + C, since 

2C = {{xi,X2) I -2 < xi < 2,X2 > 0} U {(-2,0), (2,0)}, 

C + C = 2CU (0,0). 
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On the contrary, 2C = C + C whenever C is a convex set Il24l Theorem 3.2], 

Example 2.3 Define 

El := {{xi,X2) I Xi > 0,X2 £ IR} \ {(0,X2) | \X2\ < 1}, 

E 2 := {{x\,X 2 ) I X\ < 0 ,X 2 £ IR} \ {(0,X2) I \X 2 \ < 1 }. 

The set Ei n E 2 = {(0,X2) | |^ 2 | > 1} is not nearly convex. On the contrary, Ei n E 2 is 
convex if both Ei, E 2 are convex. 

Let B(x, e) C IR" be the closed ball with radius e > 0 and centered at x, and let I be an 
index set I := {1,2,..., m} for some integer m. We use conv C for the convex hull of C. 
The interior of C is int C, the core is 

coreC := {x G C I (Vy G R”)(3e > 0) [x — ey,x + ey] C C}, 

and the relative interior is 

riC := {x G aff C | 3e > 0, (x + eB(0,1)) n (aff C) C C}. 

The recession cone of C is 

recC := {y G R" I (VA > 0) Ay + C C C}. (1) 

The lineality space of C is the largest subspace contained in rec C, see Il24l page 65] for 
more on lineality spaces. We denote the projection operator onto the set C C R" by Pq 
and the normal cone operator by Nq- 

For a set-valued mapping A : R" R”, the domain is dom A := {x G R” | Ax 7 ^ 0 }, 
the range is ran A := UxeR" ^he graph is gra A := {(x, u) G R” x R” | u G Ax}. 

A is monotone if (y{x,u) G gra A)(V(y, u) G gra A) {x — y,u — v) >0, and maximally 
monotone if there exists no monotone operator B such that gra A is a proper subset of 
gra B. 


2,2 Auxiliary results on convex sets 

Properties of convex sets play a prominent role in the paper, we need to review some key 
results. 

Fact 2.4 (Rockafellar) Eet C and D be convex subsets o/R", and let A G R. Then the following 
hold: 
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(i) ri C and cl C are convex. 

(ii) C 7 ^ 0 ^ riC 7 ^ 0. 

(iii) cl (riC) = clC. 

(iv) riC = ri(clC). 

(v) aff(riC) = affC = aff(clC). 

(vi) ri C = ri D cl C = cl D ri C C D C cl C. 

(vii) rlAC = AriC. 

(viii) ri(C + D) = ri C + ri D. 


Proof. (i)&(ii) See Il24l Theorem 6.21. (iv) 


orem 6.2], (vi) See Il24l Corollary 6.3.1], (vii) 


See 

See 


Theorem 6.3]. (v) See 11241 The 


Corollary 6.6.1]. (viii) See 


Corollary 6.6.2]. 


Fact 2.5 Il24l Theorem 6.5] Let Ci be a convex set in for i = 1,... ,m such that f] ri Ci 7 ^ 0 . 

i=l 

Then 


d(nc/) = ndc., 

i=l i=\ 


ClTlci- 


(n Q = n 

i=l i=l 

Fact 2.6 Il24l Theorem 6.1] Let Che a convex set in R”, x G ri C, and y G cl C. Then 

[x,y[C riC. 

Fact 2.7 Il24l Theorem 6 . 6 ] Let Che a convex set in R” and let A be a linear transformation from 
R” to R'”. Then 

ri(AC) = A(riC), 

and 

A(clC) C cl (AC). 


Fact 2.8 1241 Theorem 6.7] Let Cbe a convex set in R” and let A be a linear transformation from 
R” to R'” such that A“^(ri C) 7 ^ 0 . Then 

ri(A-iC) = A-i(riC), 

and 

cl(A-iC) = A-i(clC). 


5 























Fact 2.9 Theorem 8.1 & Theorem 8.2] Let C be a nonempty convex subset in R”. Then 
rec C is a convex cone and 0 G rec C. If in addition C is closed then rec C is closed. 

Fact 2.10 Il24l Theorem 8.3] Let C be a nonempty convex subset in R". Then rec(riC) = 
rec (cl C). 

Fact 2.11 Il24l Theorem 9.1] Let C be a nonempty convex subset in R" and let A be a linear 
transformation from R” to R'”. Suppose that (Vz G rec(clC) \ {0}) with Az = Owe have that 
z belongs to the lineality space of cl C. Then 

cl (AC) = A(clC), 


and 

rec A(clC) = A[rec(clC)]. 

Fact 2.12 f2M Corollary 9.1.1] Let be a family of nonempty convex subsets in R" sat¬ 

isfying the following condition: if iyi G l)(3z/ G rec(cl£j)) andJ^j^jZi = 0 then (VI G I)zi 
belongs to the lineality space of cl Ei. Then 

cl (£i + • • • + £m) = cl £i + • • • + cl £,jx, (2) 

rec[cl(£i +-h £«)] =rec(cl£i)H-hrec(cl£m)- (3) 

2.3 Auxiliary results on nearly convex sets 

Near equality introduced in [SI provides a convenient tool to study nearly convex sets 
and ranges of maximally monotone operators. 

Definition 2.13 (near equality) Eet C and D be subsets o/R". INe say that C and D are nearly 
equal, if 

cl C = cl D and ri C = ri D. (4) 

and denote this by C ^ D. 

Fact 2.14 BH Lemma 2.7] Eet Ebe a nearly convex subset o/R”, say C C £ C clC, where C is 
a convex subset o/R”. Then 

£ ~ cl £ ~ ri £ ~ conv £ ~ ri conv £ ~ C. (5) 

In particular, the following hold. 

(i) cl £ and ri £ are convex. 
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(ii) IfE^0, then ri £ 7^ 0. 

Fact 2.15 m Proposition 2.12(i),(ii),(iii)] Let Ei and £2 be nearly convex subsets o/R”. Then 

£1 ~ £2 ^ ri El = ri £2 ^ cl £1 = cl £ 2 . ( 6 ) 

Fact 2.16 im Proposition 2.5] Let A, B and C be subsets o/R” such that A ^ C and A C B C 
C. Then A ^ B ^ C. 


Fact 2.17 (See ||26l Theorem 12.41].) Let A : R” R” maximally monotone. Then dom A 
and ran A are nearly convex. 


Remark 2.18 Fact 2.17 can not be localized. Suppose that A := £ 5 ( 0 , 1 ) is the projection 


onto the unit ball in R^, which is a gradient mapping of the continuous differentiable 
convex function / : R^ ^ R given by 


fix) 


x|p /2 if ||x|| < 1 , 
x|| — 1/2 if ||x|| > 1 . 


(i) Let S := {(x,i/) G R^ | x + y > 2,x > 0,i/> 0} be open convex. The set 

ran £3(04) (S) = {(x,y) G R^ | + y^ = 1 , x > 0,y > 0} is not nearly convex. 

(ii) Let S := {(x,y) G R^ | x + y > 2,x > 0,y > 0} be closed convex. The set 
ran £ 3 ( 04 ) (S) = {(x,y) G R^ | x^ + y^ = l,x > 0,y > 0} is not nearly convex. 

We refer readers to lUlIlOlIll 12411271 for more materials on convex analysis and monotone 
operators. 


3 Characterizations and basic properties of nearly convex 
sets 


Utilizing near equality, in |3 the authors provide the following characterizations of nearly 
convex sets. 

Fact 3.1 (characterization of near convexity) HU Lemma 2.9] Let E C R". Then the follow¬ 
ing are equivalent: 

(i) £ is nearly convex. 
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(ii) E ~ conv E. 

(iii) E is nearly equal to a convex set. 

(iv) E is nearly equal to a nearly convex set. 

(v) riconvE C E. 


We now provide further characterizations of nearly convex sets. 

Theorem 3.2 Eet E be a nonempty subset o/R”. Then the following are equivalent: 


(i) E is nearly convex. 

(ii) ri E is convex and cl (ri E) = cl E. 

(iii) (Vx G riE) (Vy G E) [x,y[C riE. 

(iv) cl E is convex and ri(cl E) C ri E. 


Proof, (i)^(ii) This follows from Fact 2.14 (h^ (i) We have riE C E C E = cl (riE). 


Since ri E is convex, E is nearly convex, (ii) ^(iii) Since E is nearly convex, by Fact 2.14p) 


clE is convex. Using Fact 2.6 applied to the convex set clE we conclude that (Vx G riE) 


(Vy G clE) [x,y[C ri(clE) = riE. In particular, (Vx G riE) (Vy G E) [x,y[C riE. (iii)^(ii) 
Let X G riE and y G riE. Then [x,y] = [x,y[ U {y} C riE U riE = riE, that is riE is 
convex. It is obvious that cl (ri E) C cl E. Now we show that cl E C cl (ri E). Indeed, let 
y G clE. Then (3(y„)„eN) C E such that yn —> y. Therefore (Vx G riE) [x,yn[C riE, 
and consequently [x,y„] C cl (riE). That is, (yn)n6]N U cl(ri E), hence y G cl (riE), and 
therefore cl E C cl (ri E), as claimed, (i) ^ (iv) This follows from Fact 2.14[ |(iv) : 


(i) Since 


E is nonempty we have cl E is nonempty and convex by assumption, hence cl [ri(cl E)] = 
cl E. Now ri(cl E) C ri E C E C cl E = cl [ri(cl E)], hence ri(cl E) ~ E. Since cl E is convex 
we have ri(cl E) is convex. It follows from Fact 3.1|^iii) that E is nearly convex. ■ 


Theorem 3.3 Eet Ebea nonempty subset in R”. Then E is nearly convex if and only ifE = C U S 
where C is a nonempty convex subset o/R” and S C cl C \ ri C. 


Proof. (=>) Suppose that E is nearly convex, and notice that E = riEU(E\riE). Set 

C:=riE and S := E \ ri E C cl E \ ri E. (7) 

Since E is nearly convex, it follows from Fact |2.14] that C is nonempty and convex. More¬ 
over clE = cl (riE) = clC. Therefore E = C U S with C convex and S C clC \ riC. 
(<^) Conversely, assume that E = C U S where C is a nonempty convex subset of R” and 
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S C clC \ riC. Clearly, S C 9(clC), where 9(clC) is the relative boundary of clC. Hence 
ri £ = ri C and consequently ri £ is nonempty and convex. Moreover, since cl S C cl C, we 
have cl £ = cl (C U S) = cl C U cl S = cl C and consequently cl £ is convex. Finally notice 
that 

riC = ri£ C £ C cl£ = clC = cl (riC). (8) 


That is £ ~ ri C and hence £ is nearly convex by Fact 3.1|[iii)[ 


To study the relationship among core, interior and relative interior of a nearly convex 
set, we need two facts. 


Fact 3.4 Let C be a convex set in R”. Then intC = coreC. Moreover, if iniC 7^ 0 then 
int C = ri C. 


Proof. For the first part, see [SI Remark 2.73] or HI Proposition 6.12]. The second part is 
clear from Il24l pg 44]. ■ 

Fact 3.5 10 Proposition 2.20] Let Ebe a nearly convex subset o/R”. Then int £ = int(cl £). 

Theorem 3.6 Let Ebe a nonempty nearly convex subset in R”. Then the following hold: 


(i) core £ = int £. 

(ii) If int £ 7^ 0 then int £ = ri £. 
(hi) aff(ri£) = aff£ = aff(cl£). 


Proof. [(i)j Since £ is nonempty and nearly convex, cl£ is nonempty and convex by 
Fact [2.14 ^ Using Fact |3.5| and Fact|^ applied to the convex set cl £ we have 


int(cl£) = int£ C core£ C core (cl £) = int(cl£). 
Hence int £ = core £, as claimed. 


(9) 


(ii) Notice that Fact |3.5| gives 0 7^ int £ = int(cl £). Moreover since £ is nearly convex. 


Fact 2.14 implies that ri £ = ri(cl £). Applying Fact 3.4 to the convex set cl £ gives 


int£ = int(cl£) = ri(cl£) = ri£. 


( 10 ) 


(hi) It follows from Fact |2.14] that 


ri£ = ri(ri£) and cl(ri£) = cl£. 


( 11 ) 
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( 12 ) 


Using ( [TT| ) and Fact 2A V) applied to the convex set ri E we have 

aff(riE) = aff[cl(riE)] = aff(clE). 


Under mild assumptions, a nearly convex set is in fact convex as our next result shows. 

Definition 3.7 We say that a set E C X is relatively strictly convex if]x,y[C riE whenever 
x,y ^ E. 

Proposition 3.8 Eet E be nearly convex. Then the following hold: 


(i) If E is relatively strictly convex, then E is convex. 

(ii) If E is open, then E is convex. 

(iii) If E is closed, then E is convex. 

(iv) If for every x,y ^ E\riE, we have [x, y] C E, then E is convex. 


Proof, (i): Let x,y G E. As E is relatively strictly convex, ]x,i/[C riE C E, so [x,y] C E. 
Hence E is convex. 


(ii) For a nearly convex set E, we have ri E = ri C where C is a convex set. When E is 


open and int E 7^ 0, by Theorem 3.f li) we have E = ri E = ri C = int C is convex. Hence 
E is convex. 


(iii) Since there exists a convex set C such that C C E C cl C and E closed, we have 


E = cl C, so E is convex. 


(iv) Let X, y G E. If one of x, y is in ri E, then [x, y] C E; if both x, y G E \ ri E, then the 


assumption guarantees [x,y] C E. Hence E is convex. 


4 Calculus and relative interiors of nearly convex sets 


In this section we extend the calculus for convex sets provided in Il24l Section 6 and Sec¬ 
tion 8] to nearly convex sets. More precisely, we study the properties of images and 
pre-images of nearly convex sets under linear transforms. One distinguished feafure is 
thaf when fwo nearly convex sets are nearly equal, their linear images or linear inverse 
images are also nearly equal. We start with 
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Proposition 4.1 (i) Let E be nearly convex, and A G R. Then ri(AE) = Ari£. 

(ii) If {\fi G 1 ) Ej C R” be nearly convex, then 


ri(£i X • • • X Em) = ri£i X • • • X riE^, 
and cl (El x • • • x Em) = cl (ri Ei) x • • • x cl (ri£„^). 


Proof, (i) As E is nearly convex set, there exists a convex set C C R" such that ri £ = ri C. 


Then n(AE) = ri(AC) = A ri C = A ri E by Fact 2.4 Vii) 
|(ii)| By Fact 2.14[ we have 


ri(£i X • • • X Em) = ri[cl(Ei x • • • x Em)] = ri(cl£i x • • • x clE^) 
= ri(clEi) X • • • X ri(clEm) = riEi x • • • x riEm- 


Also by Fact 2.14 cl(Ei x • • • x Em) = cl Ei x • • • x cl E^ = cl (ri Ei) x • • • x cl (ri E„ 


Theorem 4.2 Eet E be a nearly convex set in R” and let A : R” —^ R'” be a linear transforma¬ 
tion. Then the following hold: 


(i) A(E) is nearly convex. 

(ii) AE ~ A(ri£) ~ A(clE). 
(hi) ri(AE) = AriE. 


Proof, (i) Since £ is nearly convex we have ri £ is convex. It follows from Facf |2.14' that 
cl E = cl ri E. Moreover Fact 2.7 applied to the convex set ri E implies that A(cl (ri E)) C 
clA(ri£). Therefore, 


A(riE) C A(E) C A(clE) = A(cl(riE)) C clA(riE). 


(13) 


Since A is linear and riE is convex, we conclude that A(riE) is convex, hence by ( [l^ 
A(E) is nearly convex. 


(ii) It follows from Fact |2.14] ([T^ and the fact that A(ri E) is convex that 

AE ~ A(riE). 


(14) 


To show A(riE) ~ A(clE), applying (14) to the convex set clE we obtain A(clE) 
A(ri(cl E)). Since E is nearly convex, we have ri E = ri(cl E) by Fact |2.14] hence A(cl E) 
A(ri E), and (ii)|holds. 
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By |(ii)| and Fact |2.7[ we have 

ri(AE) = ri[A(clE)] = A(ri(clE)) = A(riE). 


Remark 4.3 Theorem |4.2||^i)^(iii)| was proved in IIT3l Lemmas 2.3, 2.4], our proof is differ¬ 
ent from theirs. 

Corollary 4.4 Let (Vi ^ I) Ej be nearly convex sets in R”. Then 

ri(Ei Effi) = ri El ri Ef^. 


Proof. Apply Theorem 4.2 lii) and Proposition 4.1 'ii) with A 
where x, G R”, and E := Ei x • • • x Em. 


(\l,...,\m) ^ Liel^i 


Theorem 4.5 Eet A : R” —> R'” be a linear transformation and let E C R'” be a nearly convex 
set such that A“^(riE) 7 ^ 0 . Then 


(i) A ^E is nearly convex, 

(ii) ri(A-iE) = A-i(riE), 
(hi) cl[A-i(E)] = A-i(clE). 


Proof. As E is nearly convex, there exists a convex set C such that C C E C cl C and ri E = 
riC. Th e assurn ption A“^(riE) 7 ^ 0 is equivalen t to A~^ (ri C) 7 ^ 0 , so A“^(riclC) 7 ^ 0 
by Fact 2.4 'iv) Because A“^(riC) 7 ^ 0 , by Fact 2.8 we have cl [A“^(C)] = A“^(clC). 
Then 

A“i(C) C A“^(E) C A“^(clC) C cl[A“^(clC)] = cl[A“i(C)] 

which gives A“^(C) C A“^(E) C cl[A“^(C)], so il holds. It also follows that 
ri(A“^(E)) = ri(A“^(C)). By Fact' ' 


2.8 


ri(A-i(C)) = A-i(riC) = A-\nE). 
Therefore |(ii)| holds. |(iii)| follows from 

cl [A“^(E)] = cl [A“^(C)] = A“^(clC) = A“^(clE). 


Remark 4.6 Theorem 4.5 was proven in llT^ Theorem 2.2, Corollary 2.1]. Flowever our 
proof is different. 
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Theorem 4.7 Let A : R” —R'” be a linear transformation and let E C R'” be a nearly convex 
set such that A~^{riE) 7^ 0. Then A~^{E) ^ A“^(riE) A“^(cl£). 


Proof. First notice that since E is nearly convex we have ri E and c l E are convex and 


ri£ = ri (cl£), hence A“^(ri(cl£)) = A“^(ri£) 7^ 0. Using Fact 

ri£ = ri(ri£). 

Applying Fact |2.8| to the convex sets ri £ and cl £ we have 

A“^(ri£) = A“i(ri(ri£)) = riA“i(ri£) 
A-i(riE) = A-i(ri(clE)) = riA-i(clE). 

Since ri E and cl £ are convex we have 

A“^(riE) and A“^(clE) are convex. 


2.14 


we have 


(15) 


(16) 

(17) 


(18) 


Moreover, it follows from ( [T^ and ( [l7l > that ri A ^(riE) = ri A ^(clE). Therefore, using 
Facf l2.15l we conclude fhaf 

A-^(riE) ^ A-i(clE). (19) 


Nofice thaf A ^(ri£) C A ^(£) C A ^(clE). Therefore using ([T^ and Facf 2.16 
conclude fhaf A“^(£) ~ A“^(ri£) ~ A“^(cl£). 


we 


The nexf resulf generalizes Rockafellar's Fact 2.5 to nearly convex sets. Our proof is 
differenf from the one given in IIT^ Theorem 2.1]. 


m 

Corollary 4.8 Eet Ei be nearly convex sets in R"/or all i G I such that f] riE; 7^ 0. Then the 

i=l 


following hold: 


m 

(i) n Ei is nearly convex. 
i=l 

m m 

(ii) n ri Ei = ri H Ei. 

i=l 

ni m 

(hi) cl f] Ei= n cl Ei. 

i=l 


Proof. Define A : R” —?• R” x • • • x R” by Ax := (x,.. .,x) where x G R”. The sef 
E := El X • • • X £„ is nearly convex. The results follow by combining Theorems 4.5 4.7 
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Corollary 4.9 Let Ei and E 2 be nearly convex sets in R" such that Ei ^ E 2 and let A : R” —^ 
R”^ be a linear transformation. The following hold. 


(i) AEi ~ AE 2 , 

(ii) If A~^{riEi) 7 ^ 0, then A~^Ei ~ A~^E 2 - 


Proof. Indeed, since Ei ~ E 2 we have ri Ei = ri £ 2 - It follows from Theorem |4.2| that 

AEi ^ A(riEi) = A(riE 2 ) AE 2 . (20) 

This gives For|(ii)[ apply Theorem 4.7 ■ 


5 Recession cones of nearly convex sets 


In this section we extend several of the results for the calculus of recession cones in Il24l 
Chapter 8 ] to nearly convex sets. Intuitively, it would seem that most recession cone 
results on convex sets should hold for nearly convex sets. Unfortunately, this is not the 
case. On the positive side, we establish that rec £ of a nearly convex set E is nearly convex 
provided that span(£ — E) = rec(cl E) — rec (cl E). 

Fact 5.1 ||2ll Theorem 8.1] Eet C be a nonempty convex subset o/R”. Then recC is a convex 
cone and 0 G rec C. Moreover, 


rec C = {y G R" | y + C C C} . 


( 21 ) 


When £ is nonempty nearly convex subset of R”, the characterization ( [2T] > may fail to be 
equivalent to Q as illustrated in the following example. Let N denote the set of natural 
numbers. 


Example 5.2 Suppose that E := {(x,y) | y > x > 0} \ ( {0} x N) C R^. Notice that 
the set £ = {(x ,y) | y > x^, x > 0} is convex and E ~ E and therefore E is nearly convex 
by Fact S.lKiii) Clearly (Vy G {0} x N) y + E C E, however y ^ recE = {0}, since 
(VA G R+ \ N) Ay + E 2 E. 


Lemma 5.3 Eet E be a nonempty nearly convex subset in R". Then rec(riE) = rec(clE), in 
particular, a closed convex cone. 
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Proof. Since E is nearly convex, it follows that ri E and cl E are convex sets. Fact 2.14 gives 
ri(ri E) = ri E and cl (ri E) = cl E. Applying Fact 2.10 to the convex set ri E we conclude 
that 

rec(riE) = rec[ri(riE)] = rec[cl(riE)] = rec(clE). (22) 

As cl E is closed convex. Fact |2.9| completes the proof. ■ 

Proposition 5.4 Given (Vi G I) Ej C ]R”. Then the following hold: 


(i) rec(Ei X • • • X Ejff — rec E^ x • • * x rec Ej^i. 

(ii) If, in addition, each Ej is nearly convex, then 

rec[cl(Ei x • • • x Em)] = rec(clEi) x • • • x rec(clEm) = rec(riEi) x • • • x rec(riEm). 

Proof. This follows from the definition recession cone Q. 

|(ii)[ By[(i)|and Lemma [5^ we have 

rec[cl(Ei x • • • x Em)] = rec(clEi x • • • x clE^) 

= rec(clEi) x • • • x rec(clEm) 

= rec(riEi) x • • • x rec(riEm). 


The following resulf is folklore, and we omit its proof. 

Fact 5.5 Eet S C R” be nonempty, and K C R” be a convex cone. Then the following hold: 

(i) The set S is bounded if and only ifc\ S is bounded. 

(ii) Tor every x G S, 

aff(S) = {x} + span(S — S). (23) 

In addition, ifO G S, then spanS = aff S = span(S — S). 

(hi) span(S — S) = span(clS — clS). 

(iv) K — K = span K. 

Fact 5.6 Il2^ Theorem 8.4] Eet C be a nonempty closed convex subset in R”. Then C is bounded 
if and only if rec C = {0}. 
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Proposition 5.7 Let E be a nonempty nearly convex subset in R”. Then E is bounded if and only 
z/rec(cl£) = { 0 }. 


Proof. Since E is nearly convex, cl £ is a nonempty closed convex. Using Fad 5.5 'i) and 
Fact |5.6| applied to cl £ we conclude that 


£ is bounded ■^cl£ is bounded rec(cl£) = { 0 } . 


(24) 


The following example shows that rec(cl £) = {0} caimot be replaced by rec £ = {0}. 
Example 5.8 The almost convex set 

£ := {{X\,X2) I — 1 < < 1,X2 G R} \ {{xi,X2) I Xi = ±1,-1 < X2 < 1} 

has rec £ = {(0,0)}, but £ is unbounded. 

Lemma 5.9 Eet S be a nonempty subset o/R”. Then rec S C rec(clS). 

Proof Let y G recS and let s G clS. Then (3{sn)neK) ^ S such that s„ —> s. Since (Vn G 
N) Sn G S, it follows from that definition of rec S that (Vn G N) (VA > 0) Ay + s„ G S, 
hence Ay + s„ —> Ay + s G cl S. That is y G rec(cl S), which completes the proof. ■ 

We are now ready tor the main result of this section. 

Proposition 5.10 Eet Ebe a nonempty nearly convex subset o/R”. Suppose that 


span[rec(cl£)] = span(cl£ — cl£). 

(25) 

equivalently, 


rec(cl£) — rec(cl£) = span(£ — £). 

(26) 

Then the following hold: 



(i) ri[rec(cl£)] C rec£. 

(ii) rec £ is nearly convex. 

(hi) rec £ ~ ri[rec(cl £)]. 

(iv) rec(ri£) ~ rec £ ~ rec(cl£). 
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Proof. Observe that ([^ and are equivalent because of Fact 5i 'iii) Sn (iv) 


(i): Lety G ri[rec(clE)] C rec(clE). Then (3e > 0) such that 

B(y, e) n aff[rec(clE)] C rec(clE). 


Therefore (Vx G cl E), x + y G cl E, and 


x+ (B(y,e) n aff[rec(clE)]) CclE. 


Using Facf 5T li) (|2^ and (|28|) we have 


B(x + y, e) n aff(E) = B(x + y, e) n (x + span(E — E)) 
= X + (B(y,e) n (span(E — E))) 

= x+ (B(y, e) n aff[rec(clE)]) 

C clE. 


(27) 

(28) 


(29) 


Thai is, X + y G ri(cl E). Since E is nearly convex we have ri(cl E) = ri E, hence x + y G 
ri E C E. In particular, it follows from ( [^ thaf (Vz G E) z + y G E. Nofe thaf rec(cl E) is a 
cone, and aff[rec(clE)] is a subspace. For every A > 0, Ay G ri[rec(clE)], multiplying 
by A > 0 gives 

B(Ay, Ae) n aff[rec(clE)] C rec(clE). 

The above argumenfs show that (Vz ^ E) z + Ay ^ E. Consequently y G rec E, as claimed. 


|(ii)^ (iii) Since E is nearly convex we have cl E is a nonempty closed convex set. There¬ 
fore by Fact 2.9 and Fact 2.4 (iii) applied to the convex set rec (cl E) we have rec (cl E) 
is a nonempty closed convex cone, ri[rec(cl E)] is a nonempty convex set and 


rec(clE) = cl [rec(clE)] = cl (ri[rec(clE)]). 

It follows from Lemma 5.9 thaf rec E C rec (cl E). Therefore using we have 

ri[rec(clE)] C rec E C rec(clE) = cl[rec(clE)] = cl (ri[rec(clE)]). (30) 


Using Facf 2.14 we conclude thaf recE is nearly convex and recE ~ ri[rec(clE)], as 
claimed, (iv) It follows from (30) that 


cl (rec E) = cl [rec(clE)]. (31) 

Since rec E is nearly convex by|(ii)[ and rec (cl E) is convex by Fact 2.9 it follows from 


and Facf 2.15 thaf rec E ~ rec(cl E). Now combine with Lemma 5.3 


The following example shows thaf in Theorem 5.10 condition (251 carmot be removed. 
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Example 5.11 Suppose that E is as defined in Example 5.2 Then {0} = rec E 76 rec E = 
R+ ■ (0,1). Note that ( |^ fails because span(£ — E) = ^ {0} x R = span[rec(clE)]. 


Unfortunately, in general we do not know whether the recession cone of a nearly con¬ 
vex set is nearly convex. We leave this as an open question. 


6 Applications 


In this section, we apply results in Section and Section to study the maximality of 
a sum of maximally monotone operators and the closedness of linear images of nearly 
convex sets. 


6.1 Maximality of a sum of maximally monotone operators 


Fact 6.1 (See, e.g., Il2^ Corollary 12.44].) Eet A ; R” R” and B : R” R” maximally 
monotone such that ri dom A n ri dom B 7 ^ 0 . Then A + B is maximally monotone. 


One can apply Fact |6.1| and Corollary |4.8| to show the following well-known result. 

Theorem 6.2 Eet {Ai}-^j be finite family of maximally monotone operators from R” R” such 
that ri(dom A;) fr 0 . Then Ai + • • • + Am is maximally monotone. 


Proof. We proceed via induction. When m = 2, the proof follows from Fact 6.1 Next, 
assume that for m G N with m > 2 we have Dri(dom AJ fr 0 implies that Ai -|- • • • -|- 
Ajn is maximally monotone. Now suppose that ri(dom A;) fr 0 . By Fact 


2.17 


all i G {1,..., m -|- 1}, dom Aj is nearly convex set. Moreover, by Corollary 4.f |(ii)]~ 


for 


ri dom A; = Cifri ri dom A; n ri dom Am+i 

= ri(n]li dom A,) n ri dom Am+l 
= ridom(Ai A,„) n ridom A^+i. 


Therefore, ridovnAi fr 0 implies that n"L^ridomAf fr 0 and that ridom(Ai -|- 
• • • Am) n ridom A^+i fr 0. By the inductive hypothesis we have Ai + ■ ■ ■ + Am is 
maximally monotone. The proof then follows from applying Fact 6.1 to the maximally 
monotone operators Ai Am and Am+i- ■ 
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6.2 Further closedness results 


Theorem 6.3 Let Ebe a nonempty nearly convex subset in R” and let Abe a linear transforma¬ 
tion from R” to R'”. Suppose that (Vz G rec(cl E) \ {0}) with Az = Owe have that z belongs to 
the lineality space of cl E. Then 

cl AE = A{clE), 

and 

recA(cl£) = A(rec(clE)). 


Proof Since E is nonempty and nearly convex, it follows from Fact 2.14 that riE is a 
nonempty convex subset and cl (riE) = clE. Therefore by assumption on E we have 
(Vz G rec(cl(riE)) \ {0} = rec(clE) \ {0}) with Az = 0 we have that z belongs to the 
lineality space of cl (ri E) = cl E. Using Theorem 4.2|[ii) we have 


clAE = cl [A(riE)]. 


(32) 


Using ( [^ , Facf|2.11 applied to the nonempty convex set ri E, and Fact|2.14[ we obtain 


clAE = cl[A(riE)] = A(cl(riE)) = A(clE), 
recA(clE) = rec A(cl(riE)) = A(rec(cl (riE))) = A(rec(clE)), 


(33) 


as claimed. ■ 

As a consequence, we have: 

Corollary 6.4 Eet {Ei)i(zj be a family of nonempty nearly convex subsets in R” satisfying the 
following condition: if if/i G f)(3zf G rec(clE/)) and Yliei^i — 0 ('^^ ^ Mongs to the 

lineality space of cl Ej. Then 


cl [El + • • • + Em) — cl El + • • • + cl Em — cl (ri Ei) + • • • + cl (ri Em), 


and 

rec[cl (El +-h Em)] = rec(clEi) H-h rec(clEm) = rec(riEi) H-h rec(riEm). 


Proof Define a linear mapping A : R” x • • • x R” —^ R” by A(xi,..., Xm) := xi + • • • + x^ 
where x, G R”. The set E := Ei x • • • x Em is nearly convex in R” x • • • x R”. It suffices 
to apply Theorem |6.3[ Lemma [5^ and Proposition |5.4[ ■ 


Corollary |6.4| generalizes Fact|2.12]from convex sets to nearly convex sets. 
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7 Examples of nearly convex sets as ranges or domains of 
subdifferential operators 


It is natural to ask: is every nearly convex set a domain or a range of the subdifferen¬ 
tial of a lower semicontinuous convex function, or a domain or a range of a maximally 
monotone operator? While we cannot answer the question, we construct some interest¬ 
ing proper lower semicontinuous convex functions with prescribed domains or ranges of 
subdifferentials. Our constructions rely on the sum rule of subdifferentials for a sum of 
convex functions, while each convex function has a sub differential domain with specific 
properties. 

Recall that for a proper lower semicontinuous convex function f : R” —> ] — oo, -|-oo], its 
subdifferential at x G R” is 


a/(x) := {w G R" I (Vy G R”) {y-x,u) +f{x) < /(y)} 


when/(x) < -|-oo; and df(x) := 0 when/(x) = -|-oo. If y is continuous at x (e.g., when 
X G intdom/), then 9/(x) 7 ^ 0; if / is differentiable at x, then 9/(x) = {V/(x)}. Fact |8.2| 
provides a convenient tool to compute df. A celebrated result due to Rockafellar states 
that the subdifferential mapping df is a maximally monotone operator, see, e.g., 1261 The¬ 
orem 12.17], Il28l Theorem 3.1.11]. In Il24l page 218], Rockafellar gave a proper lower 
semicontinuous convex function whose sub differential domain is not convex. (Rockafel¬ 


lar's function is Example 7.5 when a = 1.) According to Fact 2.17 dom df must be nearly 
convex. The Fenchel conjugate f* of / is defined by 

(V X* G R”) f*{x*) := sup {(x*,x) — /(x) | x G R”}. 


The conjugate f* is a proper lower semicontinuous convex function as long as / is, and 
df* = {df)~^. The indicator function of a set C C R” is 


Lc{x) 


0 if X G C; 
-|-oo otherwise. 


A brief orientation about our main achievements in this section is as follows. We show 
that: Every open or closed convex set in R” is a domain of a sub differential mapping, so 
are their intersections under a constraint qualification; Every nearly convex set in R is a 
domain of a sub differential mapping; In R^ every polyhedral sets with its edges removed 
but keeping its vertices is a domain of a subdifferential mapping. 
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7,1 Some general results 


A set A C R” is absorbing if for every x G R" there exists Sx such that x ^ tA when 
t > Sx- Define the gauge function of A by 

Pa{x) := inf {f I f > 0 ,x G fA}. 

Then is finite-valued, nonnegative, and positive homogeneous. 

Theorem 7.1 Let C CM.” be open convex set. Then there exists a lower semicontinuous convex 
function g : R" —> ] —oo, -|-oo] such that such that ran dg = C, equivalently, dom = C. 


Proof. Take Xq G C, and let A := C — xq. Define the lower semicontinuous convex function 


g{x) 


-|-oo 


if X G A, 
otherwise. 


The convexity of g follows from that p^ is a finite-valued convex function on R”, A = 
{x G R” I pa{x) < 1} by IITSl Exercise 2.15], and that t —> is increasing and convex on 
[0,1). Since 


dg{x) 


SpAjx) 

(1-Pa{x)T 

0 


(Vx G A), 
otherwise. 


we have dom9g = A, so ran9g* = ran(9g) ^ = A. Then ran9(g* -|- (xq, •)) = A -|- xq = 

C. ■ 


Theorem 7.2 Every nonempty closed convex set C C R” is a domain or range of a subdifferential 
mapping of a proper lower semicontinuous convex function. 


Proof. The proper lower semicontinuous convex function lq has diQ = Nq so that 
dom9/c = C. Its Fenchel conjugate Uc := if has ranBuc = ran(9ic)~^ = C. ■ 

Corollary 7.3 Assume that (Vz = 1,..., m) O; C R” is open convex, that (f/j = 
1,.. .,k) Fj C R” is closed convex, and that 

(nr=iO,)n(nf=iriF,.) (34) 

Then there exists a lower semicontinuous convex function / : R" —)• ]—oo,-|-oo] such that 
dom3/ = (nf^iOi) equivalently, ran9/* = 
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Proof. Without loss of generality, we can assume 0 G nri(ny^^fy). For each O,, 

as in Theorem |7.1t define a lower semicontinuous convex function 


gi{x) 


1 

l-po,W 

+ 00 


if X G Oi, 
otherwise. 


For each Fj, as in Theorem |7.2| define a lower semicontinuous convex function ip.. For the 
lower semicontinuous convex function 


/ gl-^ - gm + -h IPy 

the constraint qualification ( [34l > guarantees that the sub differential sum rule applies, see, 
e.g., Il24l Theorem 23.8] or ||4l Corollary 16.39]. This gives 


df — dgi + • • • + dgftt + + • • • + dipj^. 


Therefore, the subdifferential operator df has 

domdf = (n"Li doma^,) n(^/=l doma^f.) = 


12 . Nearly convex sets in ]R 


Suppose that C is a nonempty nearly convex subset of R. Then eifher C is a singlefon or 
C is an inferval, and consequenfly C is convex. 

Theorem 7.4 Suppose that C is a nonempty nearly convex (hence convex) subset of R. Then 
there exists a proper lower semicontinuous convex function / : R —]—oo,+oo] such that 
domdf = C. Consequently, ran a/* = C. 


Proof We argue by cases. 

Case (i): C is closed. This covers C := (—oo, +oo), (—oo, b], [a, +oo), [a, b] where a, & G R. 
We lef / := Lq- Then df = Nq has dom df = C. 

Case (ii): C := {a, +oo) or (—oo, b). We only consider C = {a, +oo), since the arguments 
for C := ( — 00 , a) is similar. Lef 


f{x) 


^ iix>a, 
+00 otherwise. 
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Then 


0 


1 

(x —fl)^ 


if X > fl, 
otherwise. 


df(x) = 


hasdom9/= (a,+oo). 


In the remaining cases, we can and do assume a, & G IR. 
Case (hi): If C :=]a,b[ with a, & G IR and a < b, we put 


f{x) 


—a 

71 


In cos 


( nix—a) 
b—a 



+ 00 


if fl < X < &, 
otherwise. 


Then 

a/(x) = I - «) - f) ifa<x<b, 

0 otherwise. 

has dom df = {a, b). 

Case (iv): C is half-open interval. Suppose without loss of generality that C •.=\a,b] 
with fl, & G ]R and a < b. We put 


/(x) 


— (ln(x — a) 
+ 00 


otherwise. 


Then 


has dom 9/ = {a,b\. 


df(x) 


1 

fl —X 


1 

fl—b 


[0,+oo[ 

0 


if fl < X < &, 
if X = &, 

otherwise. 


(35) 


7.3 Nearly convex sets in ]R^ 

We start with a complete analysis of the classical example by Rockafellar Il24l page 218]. 
Often in literature, it only gives that dom df is not convex without details. His function 
is modified for the convenience of our later constructions. The set of nonpositive real 
numbers is R_ := {x G R | x < O}. 
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Example 7.5 Let a > 0 and define /(^i, ^ 2 ) : IR^ —t ]— 00 , + 00 ] by 


mi. ^ 2 ) : = 


max<^ a-^i2,|^2 


+00 


otherwise. 


Thena/(^i,^ 2 ) = 


0 

0 

]R_ X {1} 

R_ X {-1} 

conv I (- (0,1) I 

-Jfr'Xo),(o,-i)} 

-yrio) 

( 0 . 1 ) 


conv 


if^l <0, 

if ^1 = 0, and |^2| < a, 
if ^1 = 0, and ^2 > «, 
if ^1 = 0, and ^2 < -a, 
if ^2 = a - and 0 < ^1 < 
if -^2 = a - and 0 < 

if 0 < ^1 < and a - > l^2|, 

if 0 < ^1 < and ^2 > « - \/?i, 

if 0 < ^1 < and - ^2 > ^ - 


( 0 ,- 1 ) 

conv I (-^,0), (0,1), (0,-1) I if ^1 = a^, and ^2 = 0, 
conv {(0,1), (0,-1)} if ^1 > a^, and ^2 = 0, 

(04) 

( 0 ,- 1 ) 


if ^1 > a^, and ^2 > 0, 
if ^1 > a^, and — ^2 > 0. 


2 ^* 


a."x 


i2-\m 


ix. 


if x{ <0 and |x|| = 1, or x{ = 0 and |x|| < 1, 
if ^ < 0, and | < 1 + 2ax{, 

— a(l — IX 2 I) if x{ < 0, and max{0,1 + 2ax{} < < 1, 


^^—^^ 2 ])-a(l — IX 2 I) if x{ < 0, and — 1 < x^ < min{0, — (1 + 2ax{)}, 


ix: 


+ 00 


otherwise. 


( 36 ) 


(37) 


Consequently, 

rana/={(^i,^2)|4<0,|^2|<l}, 

and the domain of df 

doma/ = {(^1,^2) I 4 > 0,^2 e ]R} U {(0,^2) I |^2| > «} 

is almost convex but not convex. Moreover, the Fenchel conjugate of / is f* (x{, x^) 

'0 


(38) 

(39) 


(40) 
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See Appendix A for its proof. A different example is given in |17|. 



Figure 2: A Maple Il20l snapshot. Left: Plot of / with a; = 1. Right: Plot of f* with a = 1. 

Example 7.6 Let a > 0. Define / : > ] — oo, +oo] by 

max{A; — yjx\, X 2 } if X\ > 0, 


f{x\,X2) := 


+00 


otherwise. 


Then 



^(-1/2x-i/2 

4) 

if Xl 

>0, 

and X 2 < X — 



conv{(0,l). 

(-1/2x-i/2,0)} 

if Xl 

>0, 

and X 2 = X — 


a/(xi,x2) = < 

{(04)} 


if Xl 

>0, 

and X 2 > X — 


0 


if Xl 

= 0, 

and X2 < X, 



R_ X {1} 


if Xl 

= 0, 

and X2 > X, 





if Xl 

< 0. 



In particular, dom df 

= {(^ 1 ,^ 2 ) 1 

Xl > 0} \ {(0,X2) 

X2 < a} 

is neither open nor closed. 

but it is convex. The 

same holds for the range 





ran 9/ = 

{(Xi,X2) 1 Xi 

< 0,0 < X2 < 1} 

\{(a 

Xl) 

0 < X 2 < 1 }. 



See Appendix A for its proof. 
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We are finally positioned to construct proper lower semicontinuous convex functions 
on whose sub differential domains are nearly polyhedral sets but nonconvex. Recall 
that C C R” is said to be polyhedral if it can be expressed as the intersection of a finite 
family closed half spaces or hyperplanes, i.e., 

C = ( n” , {x e R" I fi{x) < 0}) n ( nf.i {x e R" | fi{x) = o}) 

where each fi is affine. 

Theorem 7.7 In R^, every polyhedral set C having a nonempty interior and having edges re¬ 
moved but keeping all its vertices is a domain of a subdifferential mapping of a proper lower semi¬ 
continuous convex function on R^. 


Proof. Each polyhedral set is closed and convex 11261 Example 2.10]. Associated each edge 
[X;,X/_|_i] of C, one can find a closed half space H; C R^ contains C and has 
in its bo und ary. By using translation, rotation and dilation for the convex function in 
Example 7.5 we can get a lower semicontinuous convex function/, : R^ —> ]—oo,+oo] 
such that domdfi = x,_i_i[. Eor each edge of the form emanating from Xj in the 

direction vf. Rj = {xj TVj | T > O}, one can find a closed half space Hj C R^ containing 
C and has Rj in its boundary. Again, by using translation, rotation, and dilation for the 
convex function in Example 7.6 we get a lower semicontinuous convex function fj : R^ —^ 
]—oo,+oo] such that domd fj = Hj \ {xy + TVj \ T > O}. Define the lower semcontinuous 
convex function / : R^ —^ ] — oo, +oo] by 


/ ^dc + J^fi + Y^fj- 

iel ie] 

As intC fz 0 , by the subdifferential sum rule we have df = 9icic + 'Liel ^fi + 'Liej dfj. 
The maximally monotone operator df has dom df = C. ■ 

Immediately from Theorem |7.7 we see that each set in Eigure|^is the sub differential 
domain of a proper lower semicontinuous convex function on R^. 
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Figure 3: A GeoGebra IflTl snapshot. Nearly convex but not convex sets 


As a concrete example, we have 

Example 7.8 Suppose that / : > ]—oo,+oo] is defined as / fi, where I : = 

{1,2,3,4} and (Vi G I) fj are defined as 

/i : ^ R : (x,y) ^ max|y2- ^^(2 + x - y), | ^(2 + x + y) 11, (41) 

/2 : ^ R : (x,y) hg max |3 — + |x| |, (42) 

/s : R^ —^ R : (x,y) max |l — a/ 1 — y, |x| |, (43) 

/4 : r 2 ^ R ; (x,y) ^ max 1^2- ^^(2 - x - y), | ^(-2 + x - y) |} . (44) 

Then / is a proper, convex, lower semicontinuous function and 

dom9/ = ran(9/)“^ = ran 9/* 


= {(x,y) I |x| < 3,|y| < 1,2 + x-y > 0,2-x-y > 0} U {(1,1), (-1,1), (3,-1), (-3,-1)}, 
as shown in Figure 
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(a)/(?i/?2) (b)doma/ 


Figure 4: The function f and dom df of Example 


7.8 


Proof Let 


✓ 

: ]R2 ^ IR : 

(x,y) ^ 

I max 

\+oo. 

and 

Re= I 

^ cos 6 


^^sin0 

Then (V(x,y) G IR^) we have. 




{(X - \y\] , 


— sin 

COS0 ) ' 


if X > 0 ; 
otherwise. 


As df 


hi^rV) = g^{R7T/i({x,y) - (- 2 , 0 ))), 
f 2 {x,y) = g3{R-n/2{{x,y) - (0,-1))), 
= giiRn/2iix,y) - ( 0 , 1 ))), 
hi^^y) = g^(R5n/4((x,y) - ( 2 , 0 ))). 


I^iei using Example 7.5 and particularly ([^, we see that 


= {(x^y) 


dom df = dom dfi 


x\ < 3, lyl < 1,2 + X 


y > 0,2 - X - y > 0} U {(1,1), (-1,1), (3, -1), (-3, -1)} . 


Using im, Proposition 16.24] we have (df) ^ = df*, which completes the proof. ■ 

We finish this section by remarking that each set in Figure [^is a sub differential domain 
of a proper lower semicontinuous convex function. For the first set, use / := ^ 5 ( 0 , 1 ) + 
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gi + g2 where each gi has dom dgi being an open convex set whose boundary consists of 
one dotted line part of the unit circle and two dotted rays tangent to the circle. For the 
second set, use / := ^b(o,i) + + S ‘2 where each gi is obtained by rotation and translation 

of Rockafellar's function, and dom By, is a closed half space with an open line segment 
on its boundary removed. 


8 Discussions and open problems 


In this paper we systematically study nearly convex sets: criteria for near convexity; topo¬ 
logical properties such as relative interior, interior, recession cone of nearly convex sets; 
formulas for the relative interiors and closures of nearly convex sets, which are linear im¬ 
age or inverse image of other nearly convex sets. Rockafellar provided the first convex 
function whose sub differential domain is not convex. To build more examples, we com¬ 
pute the sub differential and the Fenchel conjugate of an modified Rockafellar's function. 
It turns out every polyhedral set in with edges removed but keeping its vertices is a 
domain of the subdifferential mapping of a proper lower semicontinuous convex func¬ 
tion. 

Although we have constructed some proper lower semicontinuous convex functions 
whose subdifferential mappings have prescribed domain or ranges, the general problem 
is still unsolved. Let us note that 

Theorem 8.1 Let C C R” (not necessarily convex). Then there exists a monotone operator (not 
necessarily maximal monotone) A : R” =; R” such that ran A = C. 

Proof. Consider the projection operator Pq '■ R” =1 R”. Then Pq is monotone because 
gra Pc ^ P^ic is monotone by Proposition 12.19]. Clearly ran Pc = C. 


According to Theorem 12.20], a closed set C C R” is convex if and only if Pc is 
maximally monotone. Therefore, it is the maximality to force C having more structural 
properties. 

We finish the paper with three open questions: 

(i) Is every convex set a domain or range of a sub differential mapping of a proper 
lower semicontinuous convex function (or a maximally monotone operator) in R” 
with n >21 

(ii) Is every nearly convex set a domain or range of a subdifferential mapping of a 
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proper lower semicontinuous convex function (or a maximally monotone operator) 
in R” with n > 2? 

(iii) What is the intrinsic difference between the ranges of sub differentials of proper 
lower semicontinuous convex functions and the ranges of maximally monotone op¬ 
erators? 
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Appendix A 


We shall need two facts. The first one is a structural characterization of df when a convex 
function f is lower semicontinuous, and V/ is not empty. (See also Il2^ Theorem 12.67] 
for the structure of maximally monotone operators.) The second one concerns the domain 
of the Eenchel conjugate of convex functions. 

Fact 8.2 1 11241 Theorem 25.6]) Let f be a closed proper convex function such that domf has a 
non-empty interior. Then 

df(x) = cl(convS(x)) +K{x), Vx, 

where K{x) is the normal cone to domf at x (empty ifx ^ domf) and S (x) is the set of all limits 
of sequence of the form V/(xi), V f^xf), ■ ■ ■, such that f is differentiable at X;, and xi tends to x. 

Lemma 8.3 Assume that f : R” —> ]—oo,-|-oo] is proper lower semicontinuous convex function. 
If rand f is closed, then dom/* = ran 9/. 

Proof. As ran 9/ = dom9/*, by the Bronsted-Rockafellar's theorem Il23l Theorem 3.17] 
or im Proposition 16.28], we obtain ran 9/ C dom/* C cl(ran9/). Therefore, the result 
holds. ■ 

I. Proof of Example |7.5| 

Proof. Eirst, we calculate df. We argue by cases: 
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(i) CL-^x > |^2|, 0 < because /(^i,^2) = 

(ii) = 0, 1 ^ 2 ! < 9/(0,^2) = 0- Indeed, by lim( V/(xi,X 2 ) = 


does not exists. Apply Fact 


8.2 


(iii) = 0, ^2 > a; 9/(0,^2) = K- x {!}. Note that Ndom/(0,^2) = M- x {0}. When 
X 2 > a; — > Xi > 0, we have /(xi, X 2 ) = ^ 2 / so 

, lim = 

(xi,X2)^(0,f2) 

When CL — ^Jx{ > X 2 ,0 < Xi < cl^, > 0, we have f{xi, X 2 ) = x — x\^'^, so 

lim V/(xi,X2) = lim (— 

(xi,X2)^(0,?2) (^1,X2)^(0,^2) 

does not exist. Apply Fact 8^to obtain 9/(0, ^ 2 )- 


(iv) = 0, ^2 < 9/(0, ^ 2 ) = ]R- X { —1}. The arguments are similar to | (iii) 

(v) ^ 2 > X- x^> ^x> 0: 9/(^1, ^ 2 ) = {(0,1)} because /(^i, ^ 2 ) = / 2 - 

(vi) ^2 = X - x^ > > 0: Notice that/(^i,^ 2 ) = max{/i(^i,^ 2 ),/ 2 (/i,/ 2 )}, 

where fxi^i,^ 2 ) = x - ^/^ and f 2 (^ 1 ,^ 2 ) = When |^ 2 | = x - ^1 > 0, and 

^2 > 0, we have fx{^x>^ 2 ) = hence it follows from Theorem 10.31] 

that 

9/(/i/ ^ 2 ) = conv { V/i(^ 1 , ^ 2 ), V/i(^ 1 , ^ 2 )} = conv ((- 0), (0,1) j . 


(vii) —^2 = X — \/X\, > ^1 > 0: The proof is similar to the previous case with 

/(/ 1 // 2 ) = max{/ i(^i,^ 2 ),/ 2 (/i// 2 )}, where/i(^i,^ 2 ) = ol - vfe, and/ 2 (^ 1 ,^ 2 ) = 

— ^ 2 / thus 

9/(/i// 2) = conv|(-|^ri//0),(0,-l)|. 

(viii) = a/^2 = 0: We have/(^i,^2) = max {/i(^i,^2)//2(/i//2)//3(/i//2)}/where 
/i(/i// 2 ) =^- y/i// 2 (/i,/ 2 ) = I 2 and/3 (^ 1 ,^ 2 ) = -^ 2 , thus 

9/(/i// 2) = conv|(-^,0),(0,l),(0,-l)|. 


(ix) When^i > x^,f{lx,l^ = I/2I = max{^ 2 /-/ 2 }- If ^2 > 0, then/(^i,^ 2 ) = ^ 2 /so 
9 /(^i,^ 2) = {(0,1)}. If ^2 < 0,then/(^i,^2) = -^ 2 , so 9/(^i,^2) = {(0,-1)}. If 
I 2 = 0, then 9/(^1,0) = conv{(0,1), (0,-1)} = {0} x [-1,1]. 
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Next, we calculate the Fenchel conjugate of /. 


In view of (381, ran df is closed, so dom/* = ran df by Lemma 8.3 Recall fhaf 

= Xixl + X2X2—fiXi,X 2 ) 

We proceed by cases using 


(45) 


(i) xj" < 0 and x| = 1: In fhis case Xi = 0 and \x 2 \ = X 2 > x, hence f{x\,X 2 ) = X 2 - 
Therefore (^) implies thaf= X 2 “ ^2 = 0- 


(ii) < 0 and x^ = —1: In fhis case Xi = 0 and |x 2 | = —X 2 > x, hence /(xi, X 2 ) = —X 2 . 
Therefore (|45) implies fhaf/*(xj',x|) = — X 2 + X 2 = 0. 


(ill) —l/(2x) < xj” < 0 and |x|| < 1 + 2xx\\ In fhis case, this is exactly the region 
given by the set conv | (—^/O), (0,1), (0, —1) | = 9/(x^,0). Hence, by we have 
(xj‘,x|) G 9/(x^,0) so that Xi = x^, X 2 = 0 and/(xi,X 2 ) = 0. Therefore, 

f* (x^, x|) = Xixj" + X 2 X| — f{x\, X 2 ) 

= x^ • xj" + 0 • x| — 0 = (46) 

(iv) x^ < 0 and max{0,1 + 2xx^} < x| < 1: In fhis case, fhis is the region given by 

(J |conv{(—l/2xj”^^^,0), (0,1)} : 0 < Xi < x^,X 2 = x — xj^^j \ {(0,1)}. 

Then each (x{,X 2 ) G conv < (“ 2 "^'^)' (^'^) 1 some (xi, X 2 ) satisfying 0 < Xi < 


2^^ .^2 


x^, X2 = X — x\^'^. Thus, fhere exisfs A G ]0,1] such fhaf x{ = — and Xo = 1 — A. 
Therefore we have = —\x\ = — y^x{, ^/x} = ^ 

\-x% 

“2x* 


2x* 


X2 = X — ^X\ = X + 


and /(xi,X 2 ) = x — ^Jx\ = X 2 . Now ( [4^ implies fhat 


/*(x{,x|) = Xix{ + X2X| -/(Xi,X2) 

_ + (x + ^^)x| - (x + ^i^) 


Axf- 


(v) x{ < 0 and —1 < X 2 < min{0, — (1 + 2xx{)}: In fhis case, fhis is fhe region given by 


[J |conv{(— l/ 2 x^ ^^^,0), (0, —1)} : 0 < Xi < x^, —X 2 


^ = X — xj^^l 


\{(o,-i)}. 
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Then each G conv |( —2^^,0), (0, —1)| for some {xi,X 2 ) satisfying 0 < 

Xi < a^,—X 2 = a — x\^'^. As before, let A G ]0,1]. Then x^ = — ^-4^ and x% = 


— (1 — A) = A — 1. Therefore we have = —jX^ = 


1-hX^ 


2^ 


■xl, = 


l“hX' 

X2 = —{x — ^/xi) = —X — 2^' f{x\,X2) = X — -sjxx = —X2- Now ( |4^ implies 


l+x* 

2x* ' 


that 


/*(Xj,x|) = XiX^ + X2X| -/(Xi,X2) 


(l+X^) >j< / l+X^ \ / 1 + X^ \ 

- (^ + ^)^2 - (^ + 

(l+Xo)^ / I l+Xo \ / I -1 \ (l+x|)^ (l+Xn)^ /-I I 

- (^ + A^r)(^2 +1) = - «(1 + X 2 ) 




4a:J 


4x^ 


— a;( 1 — |x|| 


(47) 


i])-Q together finish the computation of f* 


Altogether, the proof is complete. ■ 

II. Proof of Example |7.6| 

Proof. We argue by cases. 

Case 1: X 2 < x — and Xi > 0. We have f{x\,X 2 ) := x — ^Jx{. When x\ > 0, 
f{x\,X 2 ) = X — -y/xi SO 9/(xi,X 2) = (—l/2x^^'^^,0); When xi = 0 and X 2 < x, /(0,X2) = 
X, /(xi, X 2 ) = X — when xi > 0, so 9/(0, X 2 ) = 0. 

Case 2: X 2 > X — ^/xi and Xi > 0. When xi > 0, f{x\,X 2 ) = X 2 , df{x\,X 2 ) = {(0,1)}; 
Whenxi = 0,9/(0,X2) = (0,1) +R_ x {0}. 

Case 3: X 2 = x — yx}. When xi > 0, 9/(xi,X2) = conv{(0,1), (—l/2x/^'^^,0)}; When 
Xi = 0, X 2 = X, 9/(0, x) = (0, 1 ) + R_ X {0}. ■ 
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